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1. Introduction 


Abstract 


In recent years, the use of diagnosing images has been increased dramatically. An en- 
try level task of diagnosing and reading Chest X-ray for radiologist but they ought to 
require a good knowledge and careful observation of anatomical principles, pathology 
and physiology for this complex reasonings. In many modern hospital’s the tremen- 
dous number of x-ray images are stored in PACS (Picture Archiving and Communica- 
tion System). The conditions of plethora been diagnosed by the sustainable number of 
chest X-Ray. Our aim to predict the thorax disease categories through deep learning 
using chest x-rays and their first-pass specialist accuracy. In a paper the main applica- 
tion that present a pathology localization framework and multi-label unified weakly 
supervised image classification that can perceive the occurrence of afterward genera- 
tion of bounding box around the consistent and multiple pathologies. Due to consider- 
ing of large image capacity we adapt Deep Convolutional Neural Network (DCNN) ar- 
chitecture for weakly-supervised object localization, different pooling strategies, vari- 
ous multi-label CNN losses and measured against a baseline of softmax regression. 
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Over a past decades the number of x-ray performed has been increased steadily. Of these, a sustainable number are chest 


x-rays used to diagnose the condition of plethora that’s includes such as Pneumonia, Edema, Effusion, Emphysema, Fibrosis, 


Hernia, Infiltration, Mass, Nodule, Pleural Thickening, Consolidation, Pneumothorax and No finding is also a category for 


non-diseased patients. Predicting the thorax disease categories through deep convolutional neural network learning in chest 


x-ray and their metadata. In the recent years a dataset was released by NIH, through image classification we try to improve 


the f1 score of disease classification [1]. In this dataset there are 30,000 unique patients over a 25,603 gray scale identically 


sized images corresponding to common thorax diseases types. 
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Generally, doctors are quite good in diagnosing, mistake can happen, and in-deep details can be left out. In a case study 
we found that 66% of the time there is a refined opinion from an original diagnose, only 12% of time the diagnose found con- 
firmed, and 21% of time the diagnose was changed completely from original diagnose [2] whenever we take second opinions. 
In a model, we order to achieve more accurate diagnose there is a reasonable sanity check to predict the diseases based on 


X-rays. 








Atelectasis Cardiomegaly Effusion Infiltration 





° 


Mass Nodule Pneumonia Pneumothorax 


Figure 1. Eight Common Thoracic disease pragmatic in chest X-ray that authenticate inspiring task of fully computerized diagnoses. 


Our inputs are 25,603 x-ray images of 1024x1024, as well as metadata on age, gender, and number of visits to the hos- 
pital. In modified residual network as well as softmax regression we feed the features to predict the output probability of 


various thorax diseases, with multi-label classification that range from normal x-ray scan to diagnosis one to many diseases. 


2. Related Work 


In epoch of deep learning hip computer vision or uses deep neural network [3], various annotated image dataset is built by 
research efforts with diverse features plays essential role on betterment of forthcoming problems, technological progresses 
and challenges definitions. The joint learning and relationship of images (chest X-rays) and manuscript (X-ray reports) we 
basically focus on it and previous generation caption utilizes Flickr8K, MS COCO and Flickr30K to represent images that’s hold 
dataset of 8000,31000 and 123000 respectively. The image is interpreted by the five rulings through Amazon Mechanical 
Turk. 

To address this difficulty, we verify and formulate the disease localization and weakly- supervised multi-label image 
classification. The VQA technique where all image captioning is depended on ImageNet a pre-trained DCNN model that al- 
ways perform well in huge number of object and for a great baseline it serves a fine-tuning model. The medical diagnosis 
domain cannot be applied on this situation. While forming the weakly labelled medical image database we have a knowledge 


of deep image localization and recognition. 
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Figure. 2. An example of chest x-ray as input 


In this paper, our aim to predict the disease in a multi-label, multi-class, image classification. In earlier only single class 
diseases were focused in x-ray Classification [4] and with specific diseases [5]. The all parts of NIH dataset are being utilized to 


get the maximum potential of our prediction in multi-label image classification. 


3. Problem Statement 


The main objective in this paper challenges are: Firstly, the accuracy rate with multi-label classification prediction. Secondly, 
creation of Deep Convolutional Neural Network (DCNN) architecture (using ImageNet) model to compare the result of soft- 
max regression/ random classifier. Thirdly, Correlation analysis between the patient’s traits and thorax diseases. 


4. Data 


The NIH dataset was released which includes 25,603 gray scale x-ray images of 1024x1024 pixels from 30,000 unique patients, 
consist the information of patients such as patient age, patient gender and number of follow up visits. 

These are the 14 common thorax diseases such as Atelectasis, Edema, Pneumonia, Cardiomegaly, Effusion, Nodule, Fibro- 
sis, Hernia, Pheumothorax, Mass, Emphysema, Pleural Thickening, Consolidation, and Infiltration. No finding is also a category 
for non-diseased patients. 

As an input feature we utilize the x-rays as well as patient’s traits with image sized 1024x1024 in gray scale channel, 
softmax regression can be used directly and in neural net we can predict the downsamples of disease categories. The complex 
challenge is to categorize a x-ray with multi-label image classification of 14 classes thorax diseases by using the dataset to full 


potential. 


5. Method 


The two different models have been used to analyze the x-rays. Firstly, the softmax regression as a baseline which give the 
probability of the 15 classes dataset of a given image. Secondly, Deep Convolutional Neural Network (DCNN) architecture 
(using ImageNet [7]) with account of metadata such as patient age, gender, etc. 


5.1. Probability of Classification and Accuracy 


Before going into a core model, let’ s discuss our approach with multi-label, multi-class dataset (any data point (x, y), we 
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have y © {0,1}145 andO < iyi < 15, any number of diseases patient could have). After the probability of 15 classes 
were obtained from both DCNN architecture and softmax regression, categories would be decided by tagged as positive and 
tagged as negative. 

Given a datapoint (x, y) € R10242 x {0, 1}15, our model predicts y* in the following way: let p € R 15each category is as- 
signed by probabilities for x. y° value of one at each index i was given where pli] top k values of p, where k =5i yi, and other 
15 —k indices is zero. Prediction of our accuracy y* for a label y was described to be y”* - y/k (how many positive categories 
were identified correctly, divided by the total number of positive categories). It is hard to use this prediction method because 
for the new patients it is not known a prior that how many diseases they have. Even so, this would be a rigorous accuracy 
permit to train model well. 

The threshold prediction strategy was tested where the probability of all classes is larger than some t marked as 1, other 
classes marked as O. The softmax probability gets the opportunity to spread over a multiple class is equally and tagged all of 


them appropriately, in practice using t = 0.15 led to a similar accuracy to a priori tagging. 


5.2. Softmax Regression 


A simple softmax regression is been implemented as a baseline. p © R15 the matrix calculation was obtained p=Wx where 
W © R15 X10242. Denoted (W) as: 


Through the optimization of cost function [6] W was calculated: 


exp (05x(i)) 


TK, exp (6;x(i) 


w)=— bar o5)1 {ye [j] = 1}log 


Following gradient 0;: Vo,J (W ) = — ye" xO(1{yOG]=1}-p(y =j]/x;6,)) 


i=1 


5.3. DCNN Unified Framework 


The pathology localization and weakly supervised multi-label image classification framework that can perceive various 
sub-sequential and pathologies bounding boxes around other pathologies. DCNN architecture consider the large image ca- 


pacity, object localization, various pooling strategies and multi-label CNN. 
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Figure. 3. DCNN unified framework and disease localization 


Our priority is to check whether there are one or more than one pathologies is present in each x-ray image then after 
locating it in network using extracted weights and activation. This challenge can be train by multi-label DCNN classification 
model. Weakly supervised object localization methods [8, 9, 10, 11] is similar to several DCNN. Network surgery done on 
various pre-trained models using ImageNet [12, 13] such as GoogleNet [14], RestNet [15], AlexNet [16] and VGGNet-16 [17] 
through classification layer and fully connected layers. We start inserting transition layer, global layer, pooling layer, predic- 
tion layer and loss layer at the termination. To find the plausible spatial location of disease is enabled by the combination of 


deep activation [9] from transition layer and prediction inner-product layer with weights. 


e Setup of Multi-label: The numerous choices of image-label has a various option to represent and select the loss func- 
tion of multi-label classification. Here, 8-dimensional label vector are define y = [y1,...,yc,...,yC],ye © {0,1},C = 7. yc 
represent the pathology image presence with respect to it while all-zero vector [0,0,0,0,0,0,0] indicates “Normal” as 


status (Not any finding). Due to multi-label classification problem there is a loss setting in regression. 


e Transition Layer: DCNN architectures has been adopt because of huge pre-trained models variety, in this layer there 
is a uniform dimension of outputs because it transform the activation function from previous layer,S X S X D,S 
€ {7, 14, 28}. D represent spatial location of dimension features (i, j), i,j © {1, ..., S}, that can be classified by dis- 
similar model settings, such as D = 1024 for GoogLeNet and D = 2048 for ResNet. It supports to allow the weights in a 
standard form via pre-trained DCNN models, further generation of heatmap in pathology localization step which is 


perilous by using this activation layer’s. 


e Multi-label Classification Loss Layer: Instead of softmax loss function for traditional multi-label classification such as 
Euclidean (EL), Hinge loss (HL) and Cross Entropy (CEL), first we use the 3 standard loss functions for regression. 
However, the instance of positive learning is difficult (pathologies with images) and rather the label of images are 


spares, means positive ”0” or negative “1”. Hence, we host the positive/negative balancing factor (BP , BN) for pa- 
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thologies and normal classes that enforce the positive learning. Such as, weighted CEL (W-CEL) follows: 
LW-CEL(f("x), 'y) = 
Bp ); —In(f(x-)) +Bn 2 —In(1-f(x-)), 


yc=1 yc=0 


|P|+|N| |P|+|N| 
where BP is agreedto |”!  whileBNisagreedto |”! — the total number of negative or ‘1’s and positive or ‘O’s are 
|P| and |N| in a group of image labels. 








6. Result and Experiment 


There is various result has been analyze from the softmax regression and the DCNN (Deep Convolutional Neural Network) 
model, it is clearly shows that the DCNN performs and give the drastically better result than the doctor diagnosis, matrix re- 


gression, softmax regression and random weight. 


e Data collection: The unified disease localization and classification framework is evaluated and validated using the 
ChestX-ray8 database. 

e Multi-level setup: There are the various choices of multi-label classification loss function and image-label represen- 
tation. The 8-dimentional label vector y = [y1, ..., yc, ..., yC ], yc€ {0, 1}, C = 8 for each image is defined. Due to this 
problem of multi-label classification definition transit into a regression-like loss setting. 

e Constructing model: In this stage, some pretrained models like AlexNet, GoogLeNet, VGGNet and ResNet. 

e Disease Localization: Due to use of activations from transition and weight from prediction layer we can calculate the 
heatmap, and also produce the B-Box for apiece pathology candidate. 

e Training and Experimentation: The training is being done by the DCNN unified framework which helps to classify the 


images in Multi-label classification. 


Table 1. Multi-label classification using DCNN 


Diverse pre-trained models initialization 


a | 0.6467 0.6927 0.6645 0.6040 oe 0.6485 0.5495 0.7427 
GoogLeNet 0.6407 0.7057 0.6877 0.6087 0,5365 0.5577 0.5591 0.7825 
VGGNet-16 0.6285 0.7085 0.6505 0.5895 0.5105 0.6557 0.5101 0.7515 


ResrNet-50 0.7169 0.8140 0.7365 0.6127 0.5607 0.7165 | aes | 0.7890 


Diverse multi-label loss function 


0.7065 0.7265 0.7353 0.6085 0.5531 0.6547 0.5165 0.7663 
W-CEL 0.7169 0.8140 0.7365 0.6127 0.5607 0.7165 0.6335 0.7890 
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7. Conclusion 


The computerized diagnostic of the radiology image database performance not has been spoken till this work. In many mod- 
ern hospital’s the tremendous number of x-ray images are stored in PACS (Picture Archiving and Communication System). 
The conditions of plethora been diagnosed by the sustainable number of chest X-Ray. Our shot to create the “hu- 
man-machine interpreted” which helps to get the comprehensive chest X-ray comparison from the tens of thousands chest 
x-ray images present in database and became a realistic methodological challenge by using the ImageNet under the DCNN 
unified Framework. In future we can improve the accuracy validation of the images and made a UI or Android application, so 
it may be user friendly to everyone. 
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